Previously, a system and method for producing synthetic single or plural word messages was developed by Bruce Baker et al. and is disclosed in U.S. Pat. No. 4,661,916 to Baker et al., issued on Apr. 28, 1987. The system was directed to a linguistic coding system and keyboard for use by people with cognitive and/or physical impairments. The coding system and associated keyboard was used to store and access messages, which included plural word messages, sentences, phrases, full names, letters, numbers, functions, or any combination thereof.
In such a system, the keyboard was coupled to a computer, or was alternately part of the stand alone entity which included a microprocessor, memory and display. The memory stored the messages for selective retrieval by the keyboard. The messages retrieved from the keyboard were then fed to a voice synthesizer, for example, which converted them through a loudspeaker to produce audible spoken messages. On this keyboard, associated with each of a plurality of keys, were polysemous (many-meaning) symbols, also known as icons. By designating selected ones of the keys and their associated symbols or icons, selected stored messages or plural word messages were accessed from the memory.
With the system described in U.S. Pat. No. 4,661,916, messages prestored in the memory could be retrieved from memory by activating a combination of symbol keys and other keys to vary the context of the polysemous symbols. Thus, a plurality of sentences could be selectively generated as a function of polysemous symbols in combination with other polysemous symbols. This allowed a user the ability to access thousands of words or messages based upon as little as one, two, or three keystrokes. Further, with symbols being polysemous, thousands of one to five symbol sequences could be generated with only a small number of keys on a keyboard. Based upon ease of use of the system, the polysemous icons or symbols utilized, and the easily memorized symbol sequence combinations, such a system became ideal for many mentally and physically challenged users for whom spelling and typing, as well as speech itself, was extremely difficult.
The system of the '916 patent allowed for an operator to go directly from thought to speech. This was possible because each key of the keyboard bore a central image or symbol which was polysemous and illustrated an important aspect of life and/or linguistic function. The keyboards could be varied depending on the intellectual level of the intended operator. Therefore, each keyboard could in itself be a language which was designed for or with a specific user. Each of the polysemous symbols were developed to be rich in associations and in combination, signal sentence or message ideas in the operator's memory. This enabled the generation of plural word or whole sentence messages by the activation of only a limited number of keys. The device allowed for the generation of many or phrases sentences or phrases and a large core vocabulary which could be easily retrieved from memory because of the ease with which the polysemous symbols on the keys portrayed the production of whole thoughts.
Sequences of icons in fixed places invoked physical memory as well as mental memory. As such sequences are learned, the body of the user tends to remember the location of key sequences in the same way a touch-typist knows the feel of the spelling of many words. After a while a user can access the keyboard even with the icons removed or covered. This situation is parallel to that of the piano player, accordion player or organist. Such access is often called automatic access. Automaticity plays a crucial role in language processing.
Thus, the aforementioned system of the Baker '916 patent provided excellent automatic access to a users "core" vocabulary. Core vocabulary items constitute approximately 75-85% of what an individual utters. Core vocabulary is the majority of the vocabulary one uses every day. If a person's speech were recorded for a five-day period, a researcher might designate a word one uses several time each day as a "core word". A word used only once in five days, however, would be a "fringe" word. Some fairly common words are fringe words for some individuals and some fairly obscure words for the general population are core words for other individuals.
Normally, seven of eight words in a typical sentence are core words for an individual and one word is typically a fringe word. In the sentence "Don't forget to tell them, no anchovies, please.", seven of the eight words are typically core words and one word is typically a fringe word. Fringe vocabulary defines vocabulary used less frequently, either chronologically or by an individual, and is relatively large when compared with core vocabularies. While seven of eight words are core words and are drawn from a relatively small pool, fringe words are a large, constantly revolving group.
If a seven-year old child with disabilities is not yet a reader, is to be mainstreamed in a typical second grade class, he/she is going to need access to a large revolving vocabulary which might be difficult to encode utilizing memorized icon sequences. If a science chapter is entitled "Let's take a walk across the desert, and what will we see?", a child may need access to ten different desert animals and five different desert plants. Able-bodied children can speak out words they are learning. They do not need to learn to spell all the specialized vocabulary words that is used in class. However, a child with significant speech and multiple impairments (SSMI) is unable to say "Gila monster" with his mouth. Therefore, this child must communicate with his communication aid or remain effectively, an observing non-participant.
Although the aforementioned Baker '916 patent provides easy and effective access to a relatively large core vocabulary, encoding and subsequently accessing these different desert animals and plants, which are typically fringe words in a vocabulary, may not be easy. A person might have to learn icon sequences for several new nouns for science class on one day and several new historical figures on a next day, etc. Encoding, entering and memorizing these new nouns might place a large burden on an individual user, classroom teacher, and family member.
Others in the field of augmentative and alternative communications (AAC) attempted to provide access to words, phrases, and sentences through spelling, through word prediction or through pictures on dynamic, graphic screens.
Spelling is often slow, difficult, and laborious for people with SSMI. Many individuals with SSMI have difficulty acquiring reading skills throughout their lifetime, and in particular, children with SSMI are late in acquiring spelling skills. Thus spelling does not represent a good option in AAC.
Typical word prediction systems function in the following manner. A user types the first letter of a word he/she wants to express. The word prediction system then presents the user on a computer screen with a numbered list of words beginning with the chosen letter. The operator visually or auditorially scans the list to see whether his/her desired word has been included. If the desired word is included, he/she selects it using the designated number. If the desired word is not in the list, he/she types the next letter. Users must then scan a second list. If the desired word is discovered in the second list, users select the designated number and the chosen word is placed in the line of text.
Dynamic screens were used which included individual words or thousands of small single meaning pictures featuring vocabulary words used in classes, for example. Sometimes these systems were arranged with special pages on which particular messages were placed under pictures which were designed to be useful in a particular environment. With these systems, a teacher could access an animal page and fifty different animals might appear. A facilitator might design a page with only ten pictures. Under each picture might be a message useful in a fast-food environment, for example.
However, such systems became extremely difficult to use for actual communication since users tended to get lost or needed to concentrate on the cognitive task of navigating through multiple vocabulary formats. Visual search tasks were very cognitively demanding. An individual may have developed automatic access to a dozen screen arrangements, but automatic access to many dozens of screens was virtually impossible. Both word prediction systems and systems employing small pictures used word prediction or the pictures to represent core and fringe vocabulary. The flaws of representing core vocabulary in such word prediction and dynamic screen systems were as follows.
Core vocabulary included words used every day. Thus, when using dynamic screens involving thousands of single meaning pictures, many keys were activated in order to access different dynamic screens, each screen containing many single meaning pictures of a particular complexity and category level, to eventually access a single core vocabulary word.
In word prediction, if the word "please" was to be accessed, the "p" key had to first be depressed. Since there were many core vocabulary words that began with "p", the user usually had to next hit the "l" key and either read through a list of words beginning with "pl" or alternatively-hit another key. The user then needed to hit the "e" key and again view a list of words beginning with "ple" to access the word "please".
The cognitive interruptions caused by such word prediction systems often increased the time necessary for language generation. Individuals often experienced the frustration of forgetting the intended utterance while reading multiple lists of words.
Similarly, in dynamic screen systems involving thousands of single meaning pictures to represent core vocabulary, multiple screens and multiple hits by a user were also necessary. This was very undesirable for a user with severe cognitive and/or physical impairments. It was not only difficult for the user to hit several different keys, but it was also very difficult for the user to navigate many different menus of pictures to assemble a sentence of many individual words. Further, since fringe vocabulary words were not commonly used, remembering how to access these words was even more difficult. Thus, the multi-menu pictorial system provided the same cognitive distractions as word prediction.